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By Chester E. Finn, Jr., and Amber M. Northern 


Reforming high schools has been one of the biggest challenges in American education 
and the achievement evidence reveals its failures. National Assessment results in grade 12 
are flat. ACT and SAT scores are flat. American PISA and TIMSS results are essentially flat. 
College remediation rates—and dropout rates—remain high. Advanced Placement (AP) 
participation is up, but success on AP exams is not—and for minority students, it’s down. 
And though high school graduation rates are up, and though it’s certainly a good thing for 
young people to acquire that credential, it's not so good when there's reason to believe it 
does not signify the level of learning that augurs success in post-high-school pursuits. 


We at the Fordham Institute have a longstanding interest in strengthening student 
achievement and school performance, and it's no secret that we're accountability hawks: 
we believe strongly that results—and growth in results—are what matter in education, and 
we've been concerned for some time about ways in which the appearance or assertion 

of improvement may conceal something far more disappointing. In that connection, 
previous Fordham studies have unmasked what we termed the “proficiency illusion,” 

the “accountability illusion,” the rise of often-questionable “credit recovery,” and the 
discrepancy between teacher-conferred grades and student performance on statewide 
assessments. 


On the upside, we've also 4A 


documented respectable—and 
authentic—achievement gains in Federal efforts to strengthen 


the early grades, particularly among academic performance via 
disadvantaged and low-achieving school accountability have 


youngsters, and eueteD of color. aver gotten maueliteaction 
But high schools, as we've noted on 


multiple occasions, remain a huge at the high school level. 
challenge. vy 


Federal efforts to strengthen 

academic performance via school accountability have never gotten much traction at 

the high school level, where—under No Child Left Behind and now the Every Student 
Succeeds Act—there’s been more emphasis on graduation rates than on student 
achievement. To their credit, most states, at one point or another, have supplemented 
those efforts by instituting exam-based requirements for students before awarding 
diplomas. These have taken the form of multisubject graduation tests—the best known 
probably being the Massachusetts MCAS exam—as well as subject-specific end-of-course 
exams (EOCs). 


Both were extensively used until just a few years ago. At their high-water mark, graduation 
tests were required by 30 states and EOCs were employed by 30 jurisdictions (there's 
double counting there, as the two types of tests overlap somewhat). Both, however, are in 
decline now. For the class of 2020, students in just 12 states will have taken a graduation 
test, and in 26 states, students will have taken one or more EOCs. 


Three factors seem to have driven that decline: the overriding push for higher graduation 
rates, which militates against anything that might get in the way; the nationwide backlash 
against testing in general; and a handful of studies indicating that requiring students 

to pass a graduation test may 

discourage them and lead to more 44 

dropouts, which is obviously bad 

for them and would also depress It’s possible that states, in 


the graduation rate, without much turning away from EOCs, are 
f itive i t e 
evidence of a positive impact on throwing = healthy baby out 


student achievement. 4 : 
with the testing bathwater. 
Yet very little prior research has 


looked at EOCs in particular. We vy 
wondered: How, exactly, do states 

employ them? And what difference, if any, do they make for student achievement and for 
graduation rates? If they cause more harm than good, states might be right to downplay 

or discard them. If, on the other hand—and unlike graduation exams—they do good 

things for kids and/or schools, it's possible that states, in turning away from EOCs, are 
throwing a healthy baby out with the testing bathwater. 


We entrusted this inquiry to Fordham’s own Adam Tyner and Lafayette College economist 
Matthew Larsen, and they've done a first-rate job, the more so considering how 
challenging it is to corral EOCs separately from other forms of testing, how tricky it is 

to determine exactly what a test is being “used for,” and how many different tests and 
states are involved and over such a long period of time. It’s also a big problem that the 
nation lacks a reliable gauge of state-by-state achievement at the twelfth-grade level—a 
challenge that the National Assessment Governing Board recently promised to address, 
but not until 2027! 


As you will see in the following pages, Tyner and Larsen learned a lot that's worth knowing 
and sharing because the implications for state (and district and school) policy and practice 
are potentially quite valuable. Probably most importantly, EOCs, properly deployed, have 
positive (albeit modest) academic benefits and do so without causing kids to drop out or 
graduation rates to falter. "In other words,” write the authors, “the key argument against 
exit exams—that they depress graduation rates—does not hold for EOCs.” Instead, these 
exams “are generally positively correlated with high school graduation rates.” Better 

still, “The more EOCs a state administers, the better is student performance on college 
entrance exams, suggesting that the positive effects of EOCs may be cumulative.” 


Nor are those the only potential benefits associated with the thoughtful use of EOCs. 
External exams are a good way for states to maintain uniform content and rigor in core 
high school courses and keep a check on the local impulse (often driven as much by 
parents as teachers or administrators) to inflate student grades. At the same time, EOCs 
can motivate students to take those courses more seriously and tend to place teachers 


and their pupils on the “same team”—for when the exam is external, the teacher becomes 
more coach than judge. 


Such exams also lend themselves to an individualized, “mastery”-based education system 
in which students proceed through their coursework at their own speed, often with the 
help of technology as well as teachers (to optimize this benefit, “end-of-unit" exams 
would be even more beneficial than the kind that are given only at the end of a semester 
or a year). 


We're surely not suggesting that states go crazy with EOCs—there's little danger of that 
happening in today’s climate anyway—but we do suggest that policymakers take seriously 
both the good that these exams can do and the potential harm from scrapping or 
softening them. And softening seems to be underway in more and more places, as states 
create detours around EOCs for kids who have trouble passing them, delay the year when 
they must actually be passed, 


or turn them into part of a 4 

student's course grade rather 

than actually requiring that We do suggest that policymakers 

kids pass them. take seriously both the good 

As we said, we're accountability that these exams can do and the 

hawks and are hence generally potential harm from scrapping 

opposed to softening. Yet or softening them. 

as Tyner and Larsen note, 

EOCs have the virtue of AA 


flexibility. States can deploy 

them in various ways: some firmer, some softer, and some simply as a source of valuable 
information for teachers, parents, school leaders, and policymakers. At a time when states 
are back in the driver's seat on school and student accountability, that’s mostly a good 
thing. But at a time when high school performance is flat, flat, flat, it seems to us that 
wise educators and policymakers alike should use every tool in their toolbox to build the 
scaffolding for major improvement. EOCs are such a tool. 


Executive Summary 


One of the great disappointments—and puzzles—of modern education reform is the 

lack of student achievement progress at the high school level, where results-based 
accountability has never been effectively targeted. In fact, despite much furor over federal 
testing requirements for public education, Washington has never required more than a 
single assessment of math, English language arts (ELA), and science skills during the entire 
four years of high school. Instead, it has focused chiefly on graduation rates—which have 
indeed risen but without evidence that actual student learning has risen with them. 


Beginning in the late 1990s, many states took it upon themselves to institute end- 
of-course exams (EOCs) at the high school level, tests specifically designed to assess 
students’ mastery of the content that various subject-matter courses covered. Their use 
exploded into the early 2010s. All told, 32 states and the District of Columbia have used 
them for accountability at some point, whether for Algebra, Biology, English, U.S. History, 
or other classes. In the last five years, however, that number has declined as some states 
have jettisoned EOCs or reduced the stakes associated with them for students and schools. 


Properly deployed, however, EOCs may help to uphold standards and rigor, incentivize 
high school students to work harder, encourage teachers to work with their pupils to 
succeed, and even support positive peer norms, as students are not graded relative to one 
other but against an external yardstick. 


Up to now, it's been difficult to determine just how states are using EOCs and whether 
their use is associated with positive outcomes. Do EOCs encourage low-performing 
students to drop out, as some research indicates that graduation exams do? Do they have 
a positive impact on achievement, as some studies of Advanced Placement exams suggest 
they do? Or are they a neutral influence, a way for states to maintain standards without the 
downsides of graduation exams? 


In the current study, Adam Tyner, Fordham’s associate director for research, and Matthew 
Larsen, assistant professor of economics at Lafayette College, investigate these and other 
questions. They provide a rich longitudinal look at state policies related to EOCs over 

the past 20 years, including patterns of use, subjects tested, and what types of stakes or 
consequences are tied to EOCs. They then estimate the effects of administering any EOCs 
in each subject on graduation rates and college entrance exam scores, as well as the effect 
of the overall number of EOCs administered. Their analysis yielded five major findings: 


FINDING 1: The use of EOCs has increased dramatically since the 
late 1990s but in the past few years has declined somewhat. 


Taking EOCs was not a common practice for students 25 years ago (Figure ES-1). In 2008, 
students in 14 states took at least one EOC that was used for accountability purposes; by 
2017, students in 30 states did so. Over the past three years, however, several states have 
moved to eliminate or reduce the number of mandatory EOCs. 


Figure ES-1. Many states adopted EOCs over the past 25 years, but this trend 
may be reversing. 
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FINDING 2: EOCs have been most widely used in math and 
science courses, but their use in ELA courses has risen fastest in 
the past decade. 


Math EOCs have been the most widely deployed (Figure ES-2). Science and ELA are 
almost tied for second. Although social studies EOCs are less common, U.S. History is the 
third most widely used EOC overall, after Biology and Algebra I. 


Figure ES-2. EOCs have most commonly been used for math, science, and 
English courses. 
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FINDING 3: Most state use of EOCs involves a mix of school and 
student accountability. 


Most states using EOCs have used them for both school and student accountability, 
though the former has been more common for almost the entire period of study (Figure 
ES-3). Uses vary greatly, however, as do the types of stakes attached to EOCs, with some 
states integrating exam results into their school accountability metrics and/or using them 
to fulfill ESSA’s requirement that states administer tests in math, ELA, and science in high 
school. 


When employed for student accountability, EOCs have most often been used as 
graduation requirements. Yet incorporating EOC results into student grades has become 
more popular over time. 


EOCs are rarely “high stakes.” In almost all cases, a single EOC is, at most, a “medium- 
stakes” test for students, accounting for perhaps a quarter of a student's final course grade 
or serving as one of several assessments among which students can choose to fulfill their 
graduation requirements. Likewise, when EOC results are included in school accountability 
metrics, they represent one among multiple outcomes. 


Figure ES-3. EOCs have often been used for a mix of school-level and student 
accountability, but school-level accountability has generally been more 
common. 
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Note: Data are approximate, due to aggregating similar EOCs and EOCs for optional 
courses, and we do not code all optional and uncommon EOCs in states with many 
EOCs. Categories are not mutually exclusive. 


Executive Summary | End-of-Course Exams and Student Outcomes 


FINDING 4: EOCs are generally positively correlated with high 
school graduation rates. 


The effects of EOCs on graduation rates are generally positive, with the impacts of math 
and ELA EOCs being statistically significant. There is a negative correlation between 
science EOCs and graduation rates, but it is not statistically significant (Figure ES-4). 


Figure ES-4. Most EOCs have a positive correlation with high school graduation 
rates. Science EOCs have a negative, though statistically insignificant, 
correlation. 
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Note: Broken outline indicates statistical significance greater than p < 0.10. N=720. 


A greater number of EOCs administered correlates generally with higher graduation rates. 
For example, the mean graduation rate for states during our study is 77 percent, but the 
predicted graduation rate for a state with nine EOCs is 85 percent. 


Figure ES-5. The number of EOCs a student takes correlates positively with the 
probability of graduation. 
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Note: The broken red line is the sample average, and the shaded areas are 90 
percent confidence intervals. Both the (blue) coefficient line and the confidence 
intervals are three-unit moving averages. N=720. 
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When analyzing the relationship that EOCs have with the graduation rates of black 

and Hispanic students, the pattern is less clear (Figure ES-6). Having a science EOC is 
associated with a 5.2 percentage point decrease in the graduation rate. Still, this effect is 
largely counteracted by the positive impacts of ELA and social studies EOCs. 


Figure ES-6. For black and Hispanic students, ELA EOCs have a positive 
correlation with high school graduation rates, while science EOCs have a 
negative correlation. 
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Note: Broken outline indicates statistical significance greater than p < 0.10. N=598. 


FINDING 5: When analyzing EOCs by subject area, there is no 
statistically significant correlation with college entrance exam 
scores, but students in states with the most EOCs appear to 
outperform other students on these exams. 


When we look at the relationship between having an EOC in a specific subject and college 
entrance exam scores in that subject, EOCs do not make a discernible difference. Yet 
the association strengthens when we examine the overall number of EOCs that states 
administer (Figure ES-7). In math, student performance on college entrance exams 
ticks up as the number of EOCs rises, although the differences only approach statistical 
significance for states with the greatest number of EOCs. Specifically, when states have 
eight or more of these exams—which six states did at some point during the study 
period—their students outperform the average college entrance exam score by four to 
five points on the SAT scale. For ELA, we find that states with three or more EOCs tend 
to slightly outperform states with no or fewer EOCs, although the differences are not 
statistically significant. 
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Figure ES-7. The more EOCs a state administers, the better students perform on 
college entrance exams, suggesting that any positive effects of EOCs may be 
cumulative. 
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Note: The broken red line is the sample average, and the shaded areas are 90 
percent confidence intervals. Both the (blue) coefficient line and the confidence 


intervals are three-unit moving averages. N=720. 


Turning to subgroup impacts, we see no evidence that EOCs harm the scores of black and 
Hispanic students, as their performance in states with EOCs is never lower than in those 
states without them. 


Tyner and Larsen draw three implications from these findings. 


States should leverage the potential benefits associated with 
external assessments without encountering the concerns raised 
about exit exams. 


Although external assessments have been shown in many studies to increase student 
learning, they have faced extensive criticism in the U.S. due to other studies showing that 
pass-fail exit exams have slowed the rise of graduation rates. EOCs have no such negative 
effects—and, in some cases, have positive impacts on these rates. In other words, the 

key argument against exit exams—that they depress graduation rates—does not hold for 
EOCGSs. 


States should consider building their high school accountability 


systems around EOCs given the suggestive evidence that such 
an approach will help them see better student outcomes. 
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When we look at the overall number of EOCs offered by states, we find strong clues 

that those that use EOCs as a central part of their high school accountability systems 
outperform other states in college entrance exam scores and graduation rates, even after 
controlling for a range of demographic factors, prior performance, and other relevant 
variables. 


States should take advantage of the different ways that EOCs 
can promote accountability at the high school level. 


The ability of states to customize the use and application of EOCs is a key strength of these 
tests. In jurisdictions that want to incentivize students to take their coursework seriously, 
EOCs can be used as graduation requirements, as part of course grades, or to augment 
the information on student transcripts. In states seeking greater quality control over high 
school coursework, EOCs can be deployed without “stakes” but with their results publicly 
reported so as to tamp down on grade inflation or abuse of credit-recovery programs. For 
states that want to set goals for schools and provide information to families about student 
progress, EOC results can be included in school accountability metrics. And for states that 
prefer a more hands-off approach, EOCs can be optional for districts. 


KREKEE 


The use of high-quality, content-linked external assessments could help push our 
education system toward mastery rather than seat time—a coveted goal, especially among 
advocates of competency-based and personalized models. That's unlikely to happen, 
however, in the absence of external measures that schools and employers can trust. 
Adroitly deployed, EOCs can play this role. 
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For almost four decades now—ever since the release of A Nation at Risk in 1983—American 
policy leaders have been committed to raising standards as an essential part of improving 
the outcomes of our public education system.' They have tried a variety of approaches, 
from boosting high school course requirements in the 1980s to introducing statewide 
assessment and accountability systems in the 1990s and the No Child Left Behind (NCLB) 
and Race to the Top (RTT) reforms in the 2000s and 2010s. 


However, despite the achievement gains made by our lowest-performing students at the 
elementary and middle school levels (particularly in math), these improvements have 
not endured through high school, according to the National Assessment of Educational 
Progress (NAEP). This is one of the great disappointments—and puzzles—of modern 
education reform. 


Yet federal accountability for student 44 
learning has never been effectively 


targeted at high schools. Although one. 
both NCLB and 2015's Every Student Federal accountability 


Succeeds Act (ESSA) require states to for student learning has 
assess students in multiple subjects never been effectively 


every year through most of elementary t . 
argeted at high schools. 
and middle school, there has never been & & 


a federal requirement for more than a 
single assessment of math, English, and WV 
science skills during the entire four years 

of high school. Under these federal laws, graduation rates are the only other reporting 
requirement that represents student academic achievement in high schools. Moreover, 

that heavy focus on graduation rates by both the federal and state governments has often 
allowed local schools and districts to determine how students meet diploma requirements, 
creating obvious moral hazard and likely prompting the development of new programs of 
questionable rigor, such as “credit recovery.” 


Politicians and educators can talk about “higher standards,” but raising standards is 
difficult when it means taking direct action of a punitive sort, whether that means closing 
a chronically failing school, retaining a third grader who cannot read, or—especially— 
denying a diploma to a high school student who dutifully sat through the required courses 
but did not learn much. 
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EXTERNAL ASSESSMENT 
& END-OF-COURSE EXAMS 


One response to these challenges is to separate the doers from the determiners—that 
is, stop letting schools and districts “grade their own papers” and allowing those grades 
to serve as the sole determinants of diploma readiness. External assessment makes this 
possible. 


This is not a new idea. Other countries have embraced the use of examinations with high 
stakes for students for years (and, in the case of China, for centuries). In the 1990s and 
2000s, many American states embraced graduation exams—most famously Massachusetts 
and its well-regarded MCAS system, a fairly rigorous examination that almost every 
student had to pass in order to receive a diploma. Yet recent years have found many 
policymakers and scholars souring on such exit exams, not just because of the political 
issues they raise but also because of some rigorous studies showing that they tend to 
encourage low-performing students to drop out without finding strong evidence that 
they improve student learning.” Given that these exams are often quite generic—testing 
low-level reading and math skills disconnected from what students are learning in their 
classes—that shouldn't be so surprising. 


Beginning in the late 1990s, states 4A 

increasingly turned to another form of 
End-of-course exams 

external assessment at the high school level: 

end-of-course exams (EOCs). This kind are connected to what 

of test is connected to what students are students are learning 

learning in their courses and is designed in their courses and 


to assess precisely the content that those ape dt 
courses are supposed to cover. In this way, ESIGNed (0 aSsess 


EOCs may help states overcome one of the precisely the content 


main challenges that arose after A Nation at that those courses are 


Risk, when the raising of course requirements 
ek supposed to cover. 
resulted in many districts altering course 


names and descriptions to evade true AA 
compliance. 


States’ use of EOCs exploded into the early 2010s. All told, 32 states and the District of 
Columbia have used them for accountability at some point, whether for Algebra, Biology, 
English, U.S. History, or other subjects. In the last five years, however, the number has 
shrunk, as some states have jettisoned EOCs or reduced the stakes associated with them 
for students and schools. 


In some cases, states use these EOCs as a type of graduation exam, though here things 
grow complicated (see EOCs versus Exit Exams, page 17). Only rarely do students have to 
pass specific tests in order to graduate, because in most cases there are alternatives, and 
in many cases students have choice in which exams and courses are used to meet the 
requirements. States have experimented with other ways of making the tests “count,” both 
for schools (in their accountability systems) and for students. Requiring that the test results 
be included in students’ final grades is one popular approach; putting the test results on 
students’ transcripts is another approach. 
15 


Properly deployed, EOCs may help to uphold standards, both the integrity of course 
definitions and the value of a high school diploma. Studies from around the world suggest 
that they may have other benefits, as well. First, they could encourage high school 
students to work harder—not an insignificant thing, given how many students report 

that they are not challenged at school.’ They also tend to put students and teachers on 

the same team: Rather than placing teachers in the uncomfortable position of deciding 
whether their students have met standards, especially high-stakes ones that will determine 
whether they can graduate, that role is accomplished externally, allowing the teacher to 
focus on helping students climb the mountain that is the exam.° External assessment may 
also support collaboration among students and positive peer norms, as students know that 
they are not being graded relative to each other but according to an external yardstick.® 


In this respect, state-mandated EOCs share some similarities with Advanced Placement 
(AP) and International Baccalaureate (IB) tests, which are now taken by nearly one-third of 
all U.S. high school students.’ Nobody questions whether “AP Physics” is really AP Physics; 
the College Board ensures that the course is the same everywhere, and the rigorous, 
anonymously graded external exam serves as a check. AP teachers are freed to focus 

their energies on helping their students learn the material and then succeed on the test. 
Because of this virtuous cycle, there is good evidence that the AP experience is linked to 
better long-term outcomes for students in terms of college going and persistence.® 


What has been difficult to determine, however, is just how states are using EOCs and 
whether their use is associated with positive outcomes for kids. Do EOCs encourage low- 
performing students to drop out, as some research indicates that graduation exams do? 
Do they have a positive impact on achievement, as some studies of AP exams suggest? 
Or are they a neutral influence—a way for states to maintain standards without the 
downsides of graduation exams? The following analysis seeks to answer these questions. 


ORGANIZATION OF THE REPORT 


In this report, we describe the patterns of EOC adoption over time, enabling readers to 
see which subjects are most commonly assessed and which types of accountability are 
most often implemented. Also, for the first time since the dramatic rise in the use of EOCs, 
we investigate the relationship between EOC policies and important student outcomes, 
including graduation rates and performance on the SAT or ACT—although this analysis is 
necessarily limited by the lack of uniform nationwide achievement measures for all high 
school students, which would enable comparisons across states.? 


In the pages that follow, we first describe national trends in the adoption of state- 
mandated EOCs, how the exams are used in school accountability systems, and how they 
“count” for students—as graduation requirements, in student grades, and so forth. Then 
we describe our methodology for an impact analysis, looking at the relationship between 
state EOC policies in the 33 states that have used these assessments over the last 20 years 
and key student outcomes, including high school graduation and performance on the SAT 
or ACT. We conclude with key takeaways for policymakers. 
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EOCs Versus Exit Exams 


Although the distinction is not absolute, it's important to distinguish EOCs from 

“exit exams,” about which much more research has been conducted. The latter are 
statewide exams that often test low-level skills and are required for high school 
graduation; critics argue that they fail to improve student learning, while encouraging 
dropping out. The research on dropouts has shown some contradictory results,’® but 
recent studies of the effect of exit exams on academic achievement have shown no 
effect on NAEP outcomes," small negative effects on SAT scores,’* and—in California— 
no effect on academic achievement.” And if exit exams make more students give up on 
school, it is possible that their negative effects could extend beyond the schoolhouse 
door." In the U.S., researchers have suggested both positive and negative effects of 
exit exams on adult outcomes such as wages, employment, and crime." Although 
some states, such as Massachusetts, have seen improvements that may be attributable 
to a strong exit exam system, enough evidence points to exit exams having negative 
effects without boosting achievement that concern about them is warranted. 


Although states can make passing EOCs a graduation requirement (essentially making 
them a type of exit exam), they can also be used in other ways for accountability 
purposes, and a substantial volume of research from the late 1990s and early 2000s 
illustrates this potential.’® For example, when they have variable outcomes (as opposed 
to pass/fail) and are delinked from graduation requirements, EOCs may provide clearer 
signals of achievement than teacher-conferred grades, while avoiding the potential 
negative effects of exit exams. Improving such signals gives students a greater stake in 
their own performance and can counteract the effects of grade inflation: A recent study 
in North Carolina finds that EOC scores are better predictors of college readiness (as 
measured by ACT scores) than grades or class attendance.” 


Concerns about negative effects of exit exams, as well as wider anxieties about testing 
and over-testing, have led some researchers to recommend that states drop them as 
graduation requirements—and some states have done so. As shown in Figure 1, just 12 
states will require students in the class of 2020 to pass exit exams, falling from a peak of 
30 states requiring them for the class of 2003. In the next section, we show how the use 
of EOCs has increased over most of that same period. 


Figure 1. Many states have recently dropped their exit exam requirements. 
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Note: Data are derived from the Digest of Education Statistics for the classes of 2001 to 2017 and 
from the anti-testing group FairTest for the class of 2020." 
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Trends in EOC Exam Adoption 


Before examining relationships between the use of end-of-course exams and any impacts 
on student achievement, let's look at state policies related to EOCs over the past 20 years.” 


FINDING 1: The use of EOCs increased dramatically since the late 
1990s but in the past few years has declined somewhat. 


As shown in Figure 2, taking EOCs was not a common practice for students 25 years ago, 
when only New York and North Carolina administered them. In the class of 2008, students 
in 14 states took at least one EOC that was used for accountability purposes (either for 
schools or for students); by the class of 2017, students in 30 states did so. In the past 
three years, however, several states (including Alabama, Delaware, and Oklahoma) have 
moved to eliminate mandatory EOCs, meaning that future student cohorts will not take 
them. (Indiana also recently eliminated EOCs, although a biology EOC is still scheduled 
to go into effect for the class of 2022.) Some other states—including Hawaii, Maryland, 
Tennessee, Texas, and Washington—have preserved some EOCs but reduced their 
number. Figure 3 shows a similar trend, this time tabulating the overall number of EOCs 
administered across all states in a given year. 


Figure 2. Many states adopted EOCs over the past 25 years, but this trend may 
be reversing. 
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Note: Multiple sources used, including state websites, media reports, and documentation 
from the Education Commission of the States and other organizations. See Section || for more 
information on the sample. 
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Figure 3. The number of EOCs in use has grown over time but recently fallen. 
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Note: Multiple sources used, including state websites, media reports, and documentation from 
the Education Commission of the States and other organizations. Data are approximate, due to 
aggregating similar EOCs and EOCs for optional courses, and we do not code all optional and 

uncommon EOCs in states with many EOCs. See Section I for more information on the sample. 


FINDING 2: EOCs have been most widely used in math and 
science courses, but their use in English courses has risen fastest 


in the past decade. 


As shown in Figure 4, math EOCs have 
been the most widely deployed. Science 
and English are almost tied for second 
most widely used. Although social 
studies EOCs are less common, U.S. 
History is the third most widely used 
EOC overall (Table 1), after Biology and 
Algebra I. (For a list of all the courses we 
categorized by subject area, see Table 
A-1in Appendix A.) Of course, the use 
of EOCs may to some extent reflect 

the number of courses in each subject 
that students typically take during high 
school. 


Table 1. EOCs are most common in 
Biology and Algebra I.?° 


Biology 22 
Algebra | 16 
U.S. History 14 
English Il 12 
Geometry 10 
Algebra II g 
English | 8 
Integrated Math | 6 
Integrated Math II 6 
Chemistry 5 
English III 5 
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Figure 4. EOCs have most commonly been used for math, science, and English 
courses. 
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Note: Multiple sources used, including state websites, media reports, and documentation from 
the Education Commission of the States and other organizations. Data are approximate, due to 
aggregating similar EOCs and EOCs for optional courses, and we do not code all optional and 

uncommon EOCs in states with many EOCs. See Section I for more information on the sample. 


FINDING 3: Most state use of EOCs involves a mix of school and 
student accountability. 


Most states using EOCs have used them for both school and student accountability, 
though the former has been more common for almost the entire period of study (Figure 5). 
States vary greatly, however, in the kinds of stakes attached to EOCs, with a few attaching 
no stakes at all.2/ Others use EOCs to hold schools accountable, where their results are 
included in state accountability metrics, creating potential incentives and consequences 
for teachers and administrators.”* Since 2016, when ESSA began to require tests of math, 
English, and science in high school, some states use EOCs to fulfill this requirement, in 
part or in full. 


States also use EOCs to hold students accountable. In New York, for example, passing five 
of the Regents Exams is required for graduation with a “Regents Diploma."”3 Other states, 
like Nevada and North Carolina, now include EOC scores in students’ course grades, rather 
than requiring a passing score on the test itself for graduation, as they once did. Indiana 
has experimented with printing EOC scores directly on student transcripts, as Arkansas 
also did in the 2000s. Still other states require students to pass the EOC in order to pass 
the course, which can turn it into a de facto graduation requirement if passing is required 
for a student to receive her diploma.” 


States also differ in the weights attached to EOC results when they are included in course 
grades. For instance, the proportion of students’ final grades that are derived from their 
EOC scores ranges from 10 to 25 percent in states that use them this way. In Georgia, 
Nevada, and Tennessee, among others, there has been a gradual phase in, with the EOC 
score comprising 10 to 15 percent of a student's final course grade in the first year, rising to 
20 or 25 percent a few years later. 
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In reality, EOCs are rarely “high stakes,” no matter how they are used. In almost all cases, a 
single EOC is, at most, a “medium-stakes” test for students, accounting for perhaps one- 
quarter of a student's final course grade or serving as one of several assessments from 
which students can choose to fulfill their graduation requirements.” Likewise, when EOC 
results are included in state accountability metrics, they represent one among multiple 
outcomes for which schools are held accountable. 


Figure 5. EOCs have often been used for a mix of school-level and student 
accountability, but school-level accountability has generally been more 
common. 
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Note: Multiple sources used, including state websites, media reports, and documentation from 
the Education Commission of the States and other organizations. Data are approximate, due to 
aggregating similar EOCs and EOCs for optional courses, and we do not code all optional and 
uncommon EOCs in states with many EOCs. Categories are not mutually exclusive. See Section 
I! for more information on the sample. 


When employed for student accountability, EOCs have most often been used as 
graduation requirements. Yet over time, the inclusion of EOC results in student grades has 
caught up with their use as graduation requirements (Figure 6). When states have adopted 
EOCs in recent years, they have more commonly incorporated EOC results into student 
grades, and Nevada and North Carolina have recently shifted from requiring passing EOCs 
for graduation to incorporating EOC scores in students’ course grades. This change in the 
types of stakes for students could be viewed as lowering the stakes associated with EOCs— 
although because many students are not on the “bubble” of passing or failing the EOCs, it 
may also have the effect of increasing the share of students to whom some stakes apply. 


A less popular form of student accountability is including EOC scores on students’ 
transcripts, as Arkansas did when the state provided paper stickers to schools to mark EOC 
proficiency levels on students’ transcripts until the state abolished its EOCs in 2015. 
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Figure 6. When used for student accountability purposes, EOCs have most 
often functioned as requirements for graduation (i.e., exit exams), although 
incorporating EOC scores into course grades has become increasingly popular. 
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Note: Multiple sources used, including state websites, media reports, and documentation from 
the Education Commission of the States and other organizations. Data are approximate, due to 
aggregating similar EOCs and EOCs for optional courses, and we do not code all optional and 
uncommon EOCs in states with many EOCs. Categories are not mutually exclusive. See Section 
Il for more information on the sample. 
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Impact Analysis: 


Data & Methods 


We use three types of data, including data on each state’s high school assessment policies, 
on our outcomes of interest (graduation rates and scores on college entrance exams), and 
various control-variable data. The unit of analysis is the “state-class"—for example, Ohio 
students who graduated in 2005 and had entered high school four years earlier. 


STATE POLICIES 


State EOC policy data were collected from published reports, official state websites, and 
communication with state departments of education (including Washington, DC).7° We 
collected information on which students took the exams in which years, the subjects tested 
on the exams, and what type of stakes or consequences were tied to them. 


Because of the decentralized nature of high school assessment policy, there are many 
idiosyncrasies and variations across states regarding how EOCs are used; in some cases, 
we had to make judgment calls about which exams to include. Generally, we include 
math EOCs (Algebra |, Geometry, Algebra II, and Integrated Math I-III), the other six 
most common EOCs (Biology, Chemistry, English I-III, and U.S. History), as well as one 
additional “wild card" EOC for each state, when applicable.”’ To verify whether we had 
accurately coded EOCs, we contacted individual state assessment officials and received 
responses from 24 of the 36 states with any EOCs during the study period.” 


The Digest of Education Statistics codes states with EOCs that are used as graduation 
requirements as having exit exams, which we combined with EOC data. We created a new 
variable indicating whether a state has exit exams but not EOCs, and we use this variable 
as acontrol in our models (for more on the differences between the two, see EOCs Versus 
Exit Exams, page 17). 


Because it is not possible to identify specific types of accountability for each individual 
year, we generally code states for all years based on the typical form(s) of accountability 
for which the EOC has been used in that state during the study period.” As all types of 
accountability (whether for students, schools, or both) may impact outcomes, either by 
incentivizing students directly or encouraging schools to get students to learn more, we 
combine these forms of accountability when analyzing the potential impacts of EOCs on 
student outcomes.°*? 
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OUTCOMES 


We examine two academic outcomes that are potentially impacted by EOC policy: high 
school graduation rates and college entrance exam scores.*" 


We compute high school graduation rates from 1990 to 2009 based on data from the 

U.S. Department of Education's (ED) Common Core of Data using the adjusted-cohort 
graduation rate method, and we include 2011 to 2016 data from the ED's EdFacts website, 
which was computed using a similar method. 


The ACT organization provided ACT data for the graduating classes of 1998 to 2017, and SAT 
data are publicly available from 1998 to 2016. We analyze which of the two college entrance 
exams had a higher participation rate for each graduating class in each state. To put these 
data on a common scale, we use an ACT-SAT score concordance endorsed by the College 
Board and the ACT organization.*4 Although the data on college entrance exams are not 
ideal, particularly given that not all students take these exams, they are the only national 
assessments that are taken by a high proportion of high schoolers on a yearly basis.*° 


CONTROL VARIABLES 


We employed various demographic controls, including the share of Asian, black, Hispanic, 
and white high-school-age population; median income; and education levels from the 
American Community Survey (ACS) in the years 2000 to 2016. For assessment outcomes, 
we include controls for the share of students in a given cohort who took the assessment 
and the demographics of exam takers (excluding racial/ethnic demographics from the 
ACS in these analyses). Although it is impossible to fully control for policy changes that 
may have occurred in conjunction with changes to EOC policies, we control for teacher- 
student ratio, per-pupil spending, and non-EOC exit exams to help control for other 
education policies.2° Additionally, we include both state and year (class) fixed effects in 
each regression. 


RESEARCH METHODS 


We use the data described above to estimate impacts of EOC policies on student 
outcomes, controlling for state and year (class) fixed effects as well as the demographic 
compositions of each class of students in each state. Our preferred model includes the 
effects of administering any EOCs in each subject—math, English language arts (ELA), 
science, and social studies—for a class, and we also include results from some models 
where we estimate the effect of the overall number of EOCs administered. For more on the 
difference-in-differences models used to estimate effects, see Appendix B. 
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Student Outcomes? 


This section examines the potential impacts of EOC policies on student academic 
outcomes, namely high school graduation rates and college entrance exam scores.?” 


GRADUATION RATES 


FINDING 4: EOCs are generally positively correlated with high 
school graduation rates. 


The effects of EOCs on graduation rates are generally positive, with the impacts of math 
and ELA EOCs statistically significant. There is a negative correlation between science 
EOCs and graduation rates, but it is not statistically significant (Figure 7). 


Figure 7. Most EOCs have a positive correlation with high school graduation 
rates. Science EOCs have a negative, though statistically insignificant, 
correlation. 
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Note: Broken outline indicates statistical significance greater than p < 0.10. N=720. 


This finding is robust to our alternative approach, which examines the effect of the number 
of EOCs on high school graduation (Figure 8). In general, a greater number of EOCs 
correlates with higher graduation rates. For example, the mean graduation rate for states 
during our study is 77 percent, but the predicted graduation rate for a state with nine EOCs 
is 85 percent. 
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Figure 8. The number of EOCs a student takes correlates positively with the 
probability of graduation. 
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Note: The broken red line is the sample average, and the shaded areas are 90 percent 
confidence intervals. Both the (blue) coefficient line and the confidence intervals are three-unit 
moving averages. N=720. 


When analyzing the relationship of EOCs to the graduation rates of black and Hispanic 
students, the pattern is less clear (Figure 9). Having a science EOC is associated with a 5.2 
percentage point decrease in the graduation rate. Still, this effect is largely counteracted 
by the positive impacts of ELA and social studies EOCs. Having an ELA EOC is associated 
with a 3.7 percentage-point increase in the graduation rate, while having a social 

studies EOC is associated with a 1.2 percentage point increase, although the latter is not 
statistically significant. 


Figure 9. For black and Hispanic students, ELA EOCs have a positive correlation 
with high school graduation rates, while science EOCs have a negative 
correlation. 
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Note: Broken outline indicates statistical significance greater than p < 0.10. N=598. 
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Analysis of the number of EOCs supports the notion that the effects of different-subject 
EOCs on black and Hispanic graduation rates negate one another; we find no correlation 
between the number of EOCs and the probability of graduating high school for black and 
Hispanic students (Figure 10). The mean graduation rate for these students is 60 percent 
during this period, and the estimated graduation rate is nearly the same no matter how 
many EOCs a state administers. 


Figure 10. The number of EOCs administered by a state does not appear to 
affect the probability of black and Hispanic students graduating from the 
state’s high schools. 
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Note: The broken red line is the sample average, and the shaded areas are 90 percent 
confidence intervals. Both the (blue) coefficient line and the confidence intervals are three-unit 
moving averages. N=598. 


In sum, EOCs are generally positively correlated with high school graduation rates, 

and taking more of them appears to boost those rates. This finding contrasts with the 
literature on high school exit exams, where some studies have found negative effects on 
graduation.*8 


COLLEGE ENTRANCE EXAMS 


FINDING 5: When analyzing EOCs by subject area, there is no 
statistically significant correlation with college entrance exam 
scores, but students in states with the most EOCs appear to 
outperform other students on these exams. 


Our preferred model examines the impact of taking at least one EOC in each of the four 
primary subject areas (i.e., math, science, ELA, and social studies). The effects of EOCs on 
college entrance exam taking are generally positive but statistically insignificant (Figures 
A-1 and A-2 in Appendix A). Likewise, correlations between EOCs and college entrance 
exam scores (both verbal and math) are generally positive but statistically insignificant 
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(Figure 11). Further, there is no clear “subject alignment”: Math EOCs have less of a positive 
correlation with math scores than with verbal scores, and ELA EOCs have no greater 
impact on verbal scores than on math scores.®? In sum, when looking at the relationship 
between having an EOC in a specific subject and college entrance exam scores in that 
subject, EOCs are not making a discernible difference. 


Figure 11. When analyzing EOCs by subject area, there are no statistically 
significant differences in college entrance exam performance between 
students in states with and without EOCs. 
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Note: Broken outline indicates statistical significance greater than p < 0.10 (no effects are 
statistically significant). The outcomes are math and verbal scores on the ACT or SAT, depending 
on which assessment is more common in the given state for a given state-cohort observation. 
N=720. 


The association between EOCs and college entrance exams strengthens when we 

examine the overall number of EOCs that states administer (Figure 12).4° In math, student 
performance on college entrance exams ticks up as the number of EOCs rises, although 
the differences only approach statistical significance for states with the greatest number of 
EOCs. Specifically, when states have eight or more EOCs, students outperform the average 
college entrance exam score by four to five points on the SAT scale.” 


For ELA, we find that states with three or more EOCs tend to slightly outperform states 
with no or fewer EOCs, although the differences are not statistically significant.” 
Altogether, these results suggest that the effect of EOCs may be cumulative and states 
that orient their high school accountability systems more around EOCs may see some 
gains in student learning, insofar as that is gauged by college entrance exams. 


The picture looks somewhat different for black and Hispanic students (Figure 13). Although 
their performance is slightly higher in states administering the most EOCs, the differences 
are small and not statistically significant. Overall, we see no evidence that EOCs harm the 
scores of black and Hispanic students, as their performance in states with EOCs is never 
lower than in those states without them (neither do we see statistically significant effects 
when looking at EOCs by subject area). 
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Figure 12. The more EOCs a state administers, the better students perform on 
college entrance exams, suggesting that any positive effects of EOCs may be 
cumulative. 
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Note: The broken red line is the sample average, and the shaded areas are 90 percent 
confidence intervals. Both the (blue) coefficient line and the confidence intervals are three-unit 


moving averages. N=720. 


Figure 13. Performance of black and Hispanic students on college entrance 
exams is highest in states with the most EOCs, but differences are statistically 
insignificant. 
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Note: The broken red line is the sample average, and the shaded areas are 90 percent 
confidence intervals. Both the (blue) coefficient line and the confidence intervals are three-unit 
moving averages. N=720. 
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As states reevaluate their high school accountability and testing policies, we draw three 
lessons from this study. 


Leverage the potential benefits associated with external 
assessments without the concerns raised about exit exams. 


This report's findings of some positive effects of EOCs on student outcomes, while limited, 
dovetail with numerous studies from the U.S. and around the world on the effects of 
external exam systems on student achievement. Yet while external assessments have been 
shown in many studies to increase student learning, they have faced extensive criticism 

in the U.S. due to other studies showing that pass-fail exit exams have slowed the rise of 
graduation rates (see EOCs Versus Exit Exams, page 17). As a 2011 blue-ribbon panel on 
the use of exit exams put it, “High school exit exam programs, as currently implemented 
in the United States, decrease the rate of high school graduation without increasing 
achievement."*? The negative impact on high school completion is concerning, but it's 
also worth keeping in mind that as graduation rates have soared in recent years, other 
measures of achievement have not kept pace.** 


Still, raising graduation rates remains 44 
a key goal for high schools in most 
states, and when we examine the The key argument against 


effects of EOCs on these rates, we exit exams—that they 


find no negative effects—and in some d d s 
cases, positive ones. In other words, epress graduation rates— 


the key argument against exit exams— does not hold for EOCs. 
that they depress graduation rates— A 
does not hold for EOCs. 


In 2014, New America’s Anne Hyslop argued against states requiring high school exit exams 
but speculated that “students can be motivated to work hard in high schools by using 
assessments toward final course grades, rather than as graduation requirements."*° In 
recent years, states such as Nevada and North Carolina have done exactly that, foregoing 
the use of the exams as graduation requirements but mandating that EOC scores be 
incorporated into students’ course grades. Policymakers should strongly consider these 
and other uses of EOCs that may result in tangible benefits to students without the 
concerns that have been raised about exit exams. 
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There’s suggestive evidence that states that build their high 
school accountability systems around EOCs see better student 
outcomes. 


Our results suggest that taking one or two EOCs in a specific subject does not have a 
measurable impact (positive or negative) on college entrance exam scores. That shouldn't 
be terribly surprising, as few interventions that do not directly address these outcomes 
have been shown to alter them.*® Further, many states only require a single EOC in a given 
subject, limiting the potential for large impacts.*” 


When we look at the overall 
number of EOCs offered YT | 
in each state, however, we We find strong clues that states 


HIGHS Ere INES Aan states that use EOCs as a central part of 


that use EOCs as a central = * orc 
part of Heir high’scnoe! their high school accountability 


accountability systems systems outperform other states 
outperform other states in in college entrance exam scores 
college entrance exam scores ° 
and graduation rates 
and graduation rates, even 
after controlling for a range vy 


of demographic factors, 

prior performance, and other relevant variables in our difference-in- differences models. 
Although differences in outcomes are nonexistent between states with just one or two 
EOCs and those with none at all, states with the most EOCs generally outperformed other 
states in college entrance exam and graduation outcomes, though the differences are 
modest and not always statistically significant.” 


Take advantage of the different ways that EOCs can promote 
accountability at the high school level. 


Unlike mandatory standardized-testing policies in elementary and middle schools, there 
is no federal policy on EOC administration. Given this freedom from federal mandates, 
states have devised a variety of ways to use them. The ability of states to customize the use 
and application of EOCs is a key strength of these tests. 


Here are some ideas for uses of EOCs that may serve specific state policy goals: 


e For states that want to incentivize students to take their coursework seriously, 
EOCs can be used as graduation requirements, as part of course grades, or to 
supplement A-to-F grades on student transcripts. States might also choose to 
award students special diplomas (or other certificates) for passing sets of state 
tests, which could serve as an important signal to employers or college admission 
officers that students have done far more than simply show up for class. Though 
AP and IB tests already offer such signals to high-performing students, EOCs can 
extend such benefits more broadly. 
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e Instates seeking greater quality control over high school course work, EOCs 
can be administered without stakes. Just having access to information on EOC 
performance can promote uniform rigor and raise flags in places where many 
students fail the EOC. Regardless of what stakes are attached, publicly reporting 
the results of EOCs can make it easier to spot grade inflation and may prevent 
misuse of credit-recovery programs meant to help students reach graduation.” 


e For states that want to set goals for schools and provide information to families 
about student academic achievement, EOC results can be included in school 
accountability metrics. 


e Forstates that prefer a more hands-off approach, they can transfer the locus of 
control to districts, making EOCs optional, and/or specify that localities can decide 
whether and how to use the tests to meet district- or school-level goals. In these 
cases, state departments of education might help vet external assessments and 
recommend high-quality options or best practices in administering them. 


Looking ahead, the use of high-quality, content-linked external assessments could 
help push our education system toward mastery rather than seat time—a coveted 
goal, especially among advocates of competency-based and personalized models. 
That's unlikely to happen, however, in the absence of external measures that schools 
and employers can trust, whether 

developed by government or by the Ad 

private sector. Adroitly deployed, 

EOCs could play this role. Indeed, High-quality, content-linked 


states such as South Dakota are external assessments could 


experimenting with them as a way ° 
for students to earn course credit, help push our education 


whether or not they have formally system toward mastery 
taken the related course.°° rather than seat time. 
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In the past five years, many states have weakened their use of EOCs. Arkansas, Delaware, 
and Oklahoma have abandoned them entirely. Tennessee and Washington have decreased 
their number. Ohio has deferred the timetable for making them count, while Pennsylvania 
has diminished the stakes they have for students. And Utah and Texas have gone so far as 
to pass laws forbidding students from being held accountable for their performance on 
the tests, although administrators and teachers may still be held accountable. 


Without access to more nuanced data about test quality and the way the EOCs are used, 
we can't say whether these changes are for better or for worse. And it remains to be seen 
whether the recent downward blip in the use of EOCs is the beginning of the end for 
them or a temporary shift as states cope with budget squeezes, testing backlash, and 

the recalibration of accountability systems in the post-NCLB era. Although we found only 
modest impacts of EOCs, other research and experience suggest that such tests can boost 
learning and hold students and schools accountable to higher standards. 


So the question is: Will states that abandon EOCs eventually live to regret it? 
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EOC SUBJECT AREAS 


We estimated the grade level (i.e., 
freshman, sophomore, etc.) students 
would take each course/EOC by examining 
relevant official and unofficial websites 

for each state. Because some states have 
many EOCs, we coded math EOCs, the 

six other most common EOCs (Biology, 
Chemistry, U.S. History, English |, English Il, 
and English Ill), and one additional EOC, if 
applicable (Table A-1). 


AP TEST TAKING AND 
PERFORMANCE 


We also examined potential impacts of 
EOCs on AP test taking and performance, 
finding that the exams are generally 
uncorrelated with these outcomes when 
looking at all students (Table A-2) or 
specifically at black and Hispanic students 
(Table A-3). 


Table A-1. Correspondence of 
Subjects and Courses 


ASU] 0) (-Yot Course 
English | 
ELA English Il 
English III 


Algebra | 

Geometry 

Algebra II 
Integrated Math | 


Math 


Integrated Math II 


Integrated Math III 
Biology 


Chemistry 


Science - 
Physics 


Physical Science 
U.S. History 


Social Studies Civics and Economics 


American Government 
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Appendix A: Additional Tables & Figures | End-of-Course Exams and Student Outcomes 


Table A-2. There are no clear patterns of effects of EOCs on AP test taking and 


passing. 


AP Subject 


Biology 


Calculus 


Chemistry 


Language 


Literature 


Note: * = p <0.10; ** = p < 0.05; and *** = p <0.01. “Pass rate” refers to the percentage of 


Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 


High pass rate 


0.0 
-0.9 
SOn 

0.1 

0.6 

0.0 

0.0 
-0.6 

0.0 

1.0 
-0.6 
-0.2 

0.3 

-1.6 
-0.8 


Estimated Effects of EOCs 


Rol (=1 al e(=) 
EOC 


0.0 
1.0 
13 

0.1 

0.6 
1.0 

0.1 

BOR 
-0.3 
13 

0.9 

0.7 
1.0 
(fare 
ihe 


0.2* 
0.1 
0.3 
0.5 
0.1 
0.1 
0.4 
0.4 
0.4 


Social 
Studies EOC 


OD 
-0.2 
-0.2 
-0.8 
09) 
-0.6 
-0.4 
-0.2 


720 
720 
720 
720 
720 
720 
720 
716 
716 
720 
720 
720 
720 
720 
720 


students earning a score of three or higher on the AP exam, and “high pass rate” refers to 
the percentage of students earning a score of four or higher on the AP exam. 
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Appendix A: Additional Tables & Figures | End-of-Course Exams and Student Outcomes 


Table A-3. There are no clear patterns of effects of EOCs on AP test taking and 


passing for black and Hispanic students. 


AP Subject 


Biology 


Calculus 


Chemistry 


Language 


Literature 


Note: * = p < 0.10; ** = p < 0.05; and *** =p <0.01. “Pass rate” refers to the percentage of 


Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 
High pass rate 
Participation rate 
Pass rate 


High pass rate 


-0.1 
-0.1 
0.0 
-0.2 
0.1 
0.3 
0.0 
= (1 
05 
0.6 
0.9 
0.0 
-0.2 
09) 
0.0 


Estimated Effects of EOCs 


Rol (=V aL e(=) 
ole 


0.1 
3.0** 
3 
0.4 
One 
-0.3 
0.1 
0.8 
-0.3 
1.3) 
0.7 
(0), 
3" 
Sess 
13** 


-1.4 


-0.6 


Social 


Studies EOC 


0.0 
0.5 
0.4 
-0.1 
hz 
0.6 
0.0 
1.8 
0.9 
-0.4 
-0.4 
-0.4 
-0.2 
SO 
-0.2 


students earning a score of three or higher on the AP exam, and "high pass rate” refers to 
the percentage of students earning a score of four or higher on the AP exam. 
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IV. Appendix A: Additional Tables & Figures | End-of-Course Exams and Student Outcomes 


Figure A-1. When analyzing EOCs by subject area, there are no statistically 
significant differences in college entrance exam taking between students in 
states with and without EOCs. 


Percentage Point Impact on 
College Entrance Exam Test Taking 


Math EOC Science EOC ELA EOC Social Studies EOC 


Note: Analysis examines binary indicators of EOCs by subject area. Broken outline indicates 
statistical significance greater than p < 0.10 (no effects are statistically significant). The outcome 
is ACT or SAT exam taking, depending on which assessment is more common in a given state for 
a given cohort. N=720. 


Figure A-2. States with EOCs have slightly higher student participation on 
college entrance exams, although the differences between states with few and 
many EOCs are small and statistically insignificant. 


=== Sample average = Predicted values 90% Confidence interval 
n 
- < 100% 
tS U 
Ei < 90% 
v7) 
= a 80% a ae 
ho 70% 

me 70% | $72 
£2 eae epnqeeampPeaweaeaceeameaameeweea=e eae ee ee eee ee ee eae ee a= ae es a= & 
oO % 60% 65% _ 
an 
2 50% 

1 2 3 4 5 6 7 8 9 

Number of EOCs 


Note: The broken red line is the sample average, and the shaded areas are 90 percent 
confidence intervals. Both the (blue) coefficient line and the confidence intervals are three-unit 
moving averages. N=720. 
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We present estimates of the effects of EOCs from two types of models. The first type 
includes a binary indicator for each EOC subject (math, science, ELA, and social studies) 
indicating whether a state had any EOCs used for accountability purposes in that subject 
area. Second, we present estimates from models showing the impact of the number of 
EOCs (up to nine) that the state administers. Note that states may have more EOCs than 
students are required to take. For example, Virginia offers EOCs in 12 courses, and students 
may select from these to fulfill the state's graduation requirements. Thus, this variable is 
best viewed as a proxy for the importance of EOCs to the state's high school accountability 
system, rather than the exact number of EOC requirements for students. 


To estimate the effects of students taking EOCs, we leverage the timing of state EOC 
requirements. To provide a plausibly causal effect, the strategy requires that the timing 
of EOC implementation is exogenous to the examined outcomes. In addition, it relies on 
the “parallel-trends” assumption, which holds that the treatment and comparison states 
must have similar trends in the outcomes prior to EOC implementation. In this way, the 
comparison states represent a satisfactory counterfactual to the EOC states absent EOC 
implementation. 


The following equation represents our preferred model for attaining estimates of the effect 
of EOCs: 


1. st= Bo + B,ELA_EOC,,+ B,Math_EOC,,+ B3Science_EOC,,+ B,SocialStudies_EOC,,+ 
B;ExitExam,,+ BeEduc,.+ B7 Econ,,+ BgDemo,.+ y;+ 6+ Est 


where Y,. is an outcome of interest (e.g., graduation rate) for state s and cohort t. 
(Subject)_EOC,, is a binary indicator equal to one if state s required any EOC in the 
corresponding subject for cohort (class) t. ExitExam,,, is a dummy for whether the state 
has anon-EOC exit examination. Educ,,, Econ,., and Demo,, are controls at the state and 
cohort level representing education (i.e., per-pupil spending and teacher-student ratio), 
socio-economic factors (i.e., median income), and demographics (i.e., share of student 
population that is Asian, black, Hispanic, or white and father’s education level). (For 
assessment outcomes, such as ACT, SAT, and AP scores, we use demographic information 
from the test-taking population instead of data from the ACS used elsewhere.) Standard 
errors are clustered at the state level. Finally, y, is a set of state fixed effects while 6, is a set 
of cohort fixed effects. B;, Bz, Bz, and B, provide the average effects of utilizing any EOC 
exams in each respective subject. 


We include additional analyses of the potential impact of the number of EOCs coded ina 
state (up to nine), represented by the following equation: 
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2. Yu=PBot LY vb: Number_EOC,.+ B,oExitExam,,+ B ,Educ,,+ B,2Econ,+ B;3;3Demo,+ 
Ver Ort €% 


where Y,, is an outcome of interest (e.g. graduation rate) for state s and cohort t. Number_ 
EOC,, is a set of binary indicators equal to one if state s required a number of EOCs 

(from one to nine) for cohort t. This estimation strategy does not assume a linear effect 

of additional EOCs, but allows us to compute a separate coefficient for each number 

of EOCs a state requires, from one to nine (with observations of states with zero EOCs 
serving as the excluded category). ExitExam,, Educ,, Econ, and Demo, are the same 

exit examination, education, economic, and demographic controls described above, with 
standard errors clustered at the state level. Finally, y, is a set of state fixed effects while 6, is 
a set of cohort fixed effects, as described above. B; through B, provide the average effect of 
utilizing the specific number of EOC exams. 


For graduation-rate and assessment-taking outcome variables, we use demographic data 
from the ACS in the years 2000 to 2016, lagged as necessary to most accurately reflect the 
demographics of the cohort. For assessment outcomes (SAT, ACT, and AP), we use the 
demographics of the test takers as controls and also include the percentage of students in 
the cohort who took the assessment as an additional control variable. 
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An additional twist with the Regents model is that many EOCs are offered and 
students choose a subset that fulfills their graduation requirements, meaning 
that some EOCs are higher stakes than others for each student. Moreover, 
students may take more than the required number of EOCs on the way to 
graduation. 


For example, the Arkansas Code (Ann. § 6-15-2009) stipulated that students 
“shall not receive credit on his or her transcript for Algebra, Geometry, Biology, 
or any other course that requires an end-of-course assessment for which the 
student has not received the requisite scale score on a general end-of-course 
assessment, until the student is identified as having participated in remediation 
through an individual academic improvement plan.” 


Until and unless employers and admission officers begin to focus on the EOC 
scores of their prospective workers and students, it is hard to imagine printing the 
results on student transcripts becoming a high-stakes proposition. 


Zinth, “End-of-Course Assessments Response,” and Domaleski, “State End-of- 
Course Testing Programs.” 


Some states allow students to select which EOCs will be used for graduation 
requirements. For example, Virginia offers EOCs in 12 courses, and students 
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EOC (Earth Science) to include in our analysis as a “wild card.” For a list of all 
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For example, in North Carolina's EOCs were used as exit exams for many years, 
but this approach was abandoned in 2010-11, with scores only incorporated into 
students’ grades from that point forward. Nevada also recently shifted their 
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used for which cohorts of students and categorize the type of accountability that 
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When examining AP performance, we dropped observations representing 
fewer than 50 black and Hispanic students (combined) from the class that 
took the given AP exam in that state. When examining high school graduation, 
we excluded observations with fewer than 1,000 black and Hispanic students 
(combined) estimated to be in the cohort. 
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including state-level fixed effects in each analysis, and controlling for the share of 
students who participate in the exam in each state-cohort observation. 


We create a binary indicator for exit exams that are not EOCs by pulling data 
from the Digest of Education Statistics and removing cases where state-class 
observations took EOCs that were graduation requirements. 


We also run tests assessing potential correlations of the administration of EOCs 
with AP test taking and success, finding no clear patterns. These results are 
presented in Appendix A. 


That said, prior analyses of exit exams include some EOCs, and the present 
analysis includes EOCs used as graduation requirements (see EOCs Versus Exit 
Exams, page 17). 


Ideally, the study could have looked individually at the number of EOCs in each 
subject, but there is not sufficient variation in the number of EOCs by subject to 
conduct such an analysis. Instead, we attempted to strike a balance between a 
focus on subjects and a focus on the number of EOCs by estimating models that 
focused on the former separately from models that focused on the latter. 


As described in Section II, these models do not take the relationship between 

the number of EOCs and student outcomes to be linear but, rather, compute a 
separate coefficient for each quantity of EOCs the students in the analyzed state- 
class took. We smooth the curves with a three-unit moving average. 


Six states were coded as having eight or nine EOCs at some point during the 
study period. 


To see the small but statistically insignificant positive effects of EOCs on college 
entrance exam taking, see Figure A-1 in Appendix A. 
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